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Abstract: The authors propose a risks evaluation model for research projects. The model is 
based on fuzzy inference. The knowledge base for fuzzy process is built with a causal and 
cognitive map of risks. The map was especially developed for research projects , taken into 
account their typical lifecycle. The model was applied to an e-testing research project: the 
probability of not obtaining quality results was computed considering the over-budget sum and 
the quality level of research idea. The computed risk is situated on the highest level in risks 
map. A software system for evaluating risks in e-testing research project was also developed. 
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1. Introduction 

Identifying, prioritizing and treating risks represent common management activities. 
For a long time, hazard risks, as well as financial ones have been actively managed. But, the 
variety, number and interactions between risks are continually increasing. The operational 
and strategic risks have increased due to the failure of the control mechanisms in a very 
dynamic business environment. In these circumstances, the organizations admit the 
importance of managing all risks, both the standard and the new risks. 

Different organizations, such as: rating agencies, stock exchanges, institutional 
investors, shareholders, and corporate governance induced an external pressure to the 
company management for managing risks systematically and comprehensively. One solution 
is to adopt the portfolio approach, the company management considering the portfolio risk 
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as the risk to the entire organization. The risks are managed considering the implications for 
the whole company, in a holistic approach. 

There is a growing tendency to quantify risks. The risk quantification allows 
managers to develop "what if" scenarios and make informed decisions. Advances in 
technology and expertise have made the quantification possible. Despite real advances, 
there will always remain risks that are not easily quantifiable, such as those related to 
human intervention and the newer ones. There is a continuing effort to quantify the portfolio 
risks, based on the individual risks and the interactions quantification. This can be extremely 
difficult if a high degree of precision is required. But, this is not usually the case. 

Over time and with practice, companies become more familiar with and more 
capable of managing risks, and even seeking out opportunities to assume risks. Companies 
understood that informed risk-taking is a means to competitive advantage. 

2. Research projects management 

The research project management is full of uncertainty and complexity. Research has 
elements of creativity and innovation and accurate prediction of the research outcome is 
therefore very difficult. It is the research project manager job to manage both the 
complexities stemming from the culture(s) of resea rc hers/ research work and the 
uncertainties associated with generating research results [4] 3 . 

The research project managers should make the following statement to the project 
team members: "If you do not have several failures, you are not doing a good job" [7]. 
Researchers acting safe are more likely to produce conservative and expected results. In 
order to obtain innovative results, the researchers should have a risk-taking behavior, 
increasing the probability of failure. This behavior should be a characteristic at the research 
system, even at the individual level, it is expected that the researcher will seek to avoid 
failure. In the majority of research projects, the purpose of project management is also to 
avoid such failures. It is an apparent conflict between the need for predictability of project 
output, "on time" and "on budget" and the unpredictability of research outcome and new 
research opportunities arising in the course of the project. Usually, the quality of output may 
improve if deviations from plan are allowed. 

The researchers ask a large degree of autonomy in their work and democracy in 
decision making. They co-operate in a research project, but, in the same time, they are 
strongly competing each others to obtain credit for the results generated in the project, such 
as: authorship of conference contributions or articles, patents. This competition may lead to 
conflict between the joint goals of the co-operation and individual goals of researchers. 

In addition, the relationship between the research project manager and the project 
participants is characterized by an asymmetric distribution of knowledge where individual 
researchers know a lot more about the potential - negative and positive - of their research 
contributions than the project manager does ([4]). 

3. Risks modeling methods 

According the manner in which the calculations are carried out, there are analytic 
and simulation methods for risk modeling. The analytic methods require a set of 
assumptions, especially related to the probability distributions. The simulation methods 
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require a large number of "trials" to approximate an answer. They are relatively robust and 
flexible, can accommodate complex relationships and depend less on simplifying 
assumptions and standardized probability distributions. Considering the manner in which the 
relationships among variables are represented, there are statistical and structural methods. 
The stotisticol methods are based on observed statistical qualities of random variables 
without regard to cause/effect relationships while the structural methods are based on 
explicit cause/effect relationships. Figure 1 presents the most important risk models 
developed using this methods, with their advantages. 
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Figure 1 . The risk model classes (source: [2]) 


The methods used to model the risks are usually customized according to the 
specific risks which occurred in the company. There are a wide variety of methods that can 
be applied to model risks. According the extent to which they rely on historical data or expert 
input ([2]), they are lying in a continuum of the sources information (figure 2). 
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Although there are numerous techniques for risk modeling, more and more experts 
are afraid to use them for making precise risk estimations. Fuzzy representations are a 
solution to obtain realistic risks assessments in projects management [3], within certain 
limits. A fuzzy abordation for risk modeling can be improved by considering causal and 
cognitive risks mapping [1], which are diagrams reflecting cause-effect relations within 
projects. 

4. Fuzzy Model for Risk Evaluation in Research Projects 

The proposed fuzzy model brings a solid contribution to risk management by 
adapting existent techniques in risks evaluation to research projects. The model has two 
important stages: risk identification from an expert database, using risk maps and building 
model components for fuzzy inference. The model allows risk quantification by knowing the 
crisp values of risk sources. Thanks to fuzzy logics mechanisms, the result has a higher 
estimation value. 

4.1. Risks Identification using Causal and Cognitive Maps 


A common approach in project risks identification is considering the risks source: 
management, cost, technology, production, environment or schedule. The fuzzy model 
considers risks not only in correlation with their source, but in correlation with project 
lifecycle, too [1]. "While it is futile to try to eliminate risk, and questionable to try to 
minimize it, it is essential that the risks taken be the right risks." [5] 

It si known that research projects are highly risky: in research project, the added 
value should be as high as possible and should be obtained as quickly as possible. [8] For 
better managing risks, it is useful to create a causal and cognitive map of risks, based on 
experts experience [1]. This map describes the propagation of risks throughout the project. 
Risks occurred at a certain moment of project lifecycle (see Fig. 3) will create other risks in 
the following moments. Risks are felt during all phases of a research project: idea 
conceptualization, project proposal development, project funding source, project initiation, 
project execution and project closing down. 
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Figure 3. Research Project Lifecycle 


20 



JOURNAL 

applied Quantitative Methods in Enterprises 

quantitative Behavior Analysis under Risk an Uncertainty 

METHODS 7 7 


Starting from the research project lifecycle (Fig. 4), following risks are identified: 

• environmental risks: lack of interest on the market, precarious economic situation, 
unfavorable legislation; 

• management risks: unrealistic duration estimation, poor negotiation capacities, poor 
planning, unclear objectives, poor communication, poor control, misunderstood 
overall vision, behind- schedule risk, acceptance of a poor idea, loss of a good idea; 

• financial risks: unrealistic budget estimation, over-budget risk; 

• production & technology: poor innovation capacities, lack of experienced 

collaborators, multidisciplinary implications, lack of quality results, low embedded 
quality of the idea; 



Figure 4. Risks Propagation Map in Research Projects 


After creating the risks propagation map, it is also useful to create a risks register or 
a risks log, in which each identified risk from the map should have a code, a name, a 
description and a source (type). A fragment of such a register is shown in Table 1. The 
identified risks are the knowledge base for creating the fuzzy model rules. 


Table 1. Fragment of Risks Register in Research Projects 


Risk 

Code 

Risk Name 

Risk Description 

Risk Type 

RSK01 

Low embedded quality of the 
idea 

The accepted idea can't be properly 
developed. 

Production & 
Technology 

RSK02 

Over-budget risk 

The project is behind schedule and 
extra budget is necessary for finishing 
the research. 

Financial 
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4.2. Model Components for Fuzzy Inference 


The proposed model for risk evaluation in research projects has the typical 
components of a fuzzy model [9]: input variables, output variable, fuzzy rules. The rules used 
in fuzzy risks modelling are built on the two well-known concepts from risks management 
(probability of risk occurance and impact of it on project development) and on primary 
causes of risks, identified in the risks map. The model can be generally stated as: „The more 
over-budget is and the more embedded quality of the research idea, the lower the degree of 
risk encountered in the research project." The risk situated at the highest level of the map 
was the only one taken into account when stating the model, because of the transitivity 
principle in risk propagation chain. In fact, the model consists from a set of rules used in 
defining project risks, which are incorporated in "Lack of quality results" risk. The model can 
be applied to calculate the value of any risk represented in risks map, using numerical values 
of its factors. An interesting situation, specific to research projects, is the fact that the 
presence of an inner risk (over-budget) diminishes the over-all risk of having poor quality 
results. 

Usually fuzzy models are used in decision making and they offer two types of 
answers: the risk can be either accepted or rejected [6]. The proposed model offers only a 
quantitative value of risk, because the decision of accepting the risk is taken by the human 
agent: project manager, risks manager or any other stakeholder. In conclusion, the output of 
the developed model isn't a form of decision, but an important parameter to make a proper 
decision. The model components are further described, using fuzzy formalization [9]. 

4.2.1. Input Variables 

The model has two forms of input variables: input functions and input constants. 

Input functions have the form of: 

P(Rsk) = probabily of Rsk occurance 

l(Rsk) = impact of Rsk on research project 

where Rsk = considered risk code 

They are described in Table 2, according to fuzzy logics concepts. 


Table 2. Input Variables Description in Risk Analysis Model 


Fuzzy Variable 
Name 

Universe of 
Discourse 

Linguistic Grades 

P(Rsk) 

[0,100] % 

VL(very low), L(low),M(medium), H(high), VH(very 
high) 

l(Rsk) 

[0, 50] 

VL(very low), L(ow),M(medium), H(high), VH(very 
high) 


Input constants have the form of: 

RskCausel = cause 1 of Rsk occurance 
RskCause2= cause 2 of Rsk occurance 
where Rsk = considered risk code 

They are described in the same manner as input functions, the only difference being 
the defined universe of discource: it is specific to each identified cause. 
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4.2.2. Output Variables 

The output variable is the value for an identified risk and is notated as: 

V(Rsk) = quantitative value of Rsk, where Rsk = considered risk code 

It is described in Table 3 and graphically represented in Fig. 5. In fact, both input 
and output variables can be graphically represented by fuzzy sets: on Ox axis the value of 
fuzzy variable is represented and on Oy axis the value of /u (function of belonging to a 
fuzzy set) is represented. The "triangles" are fuzzy sets. 


Table 3. Output Variables Description in Risk Analysis Model 


Fuzzy Variable 
Name 

Universe of 
Discourse 

Linguistic Grades 

V(Rsk) 

[0, 10] 

VL(very low), L(ow),M(medium), H(high), 
VH(very high) 



Figure 5. Fuzzy Sets Representation for Risk Value in Research Projects 


4.2.3. Model Rules 

The risk evaluation model consists from a set of predefined rules for establishing 
risks value in research projects. These inference rules are mentioned in Table 4. The 
connective used to bind conditions in rules is "and". Besides the linguistic values of model 
variables (VL, L, M, H and VH), some restrictors are used: 

• "somewhat" = %[ju 

• "very"= // 2 

where jU is the function showing if a numeric value belongs to a fuzzy set and it has values 
between 0 and 1 (a greater value shows a stronger membership). 


Table 4. Inference Rules for Analysis Risk "Rsk 11 in Research Projects 


P(Rsk)/ 

l(Rsk) 

VL 

L 

M 

H 

VH 

VL 

VL 

somewhat VL 

L 

somewhat M 

very H 

L 

L 

L 

L 

H 

VH 

M 

M 

M 

M 

H 

VH 

H 

H 

H 

H 

very H 

VH 

VH 

VH 

VH 

VH 

H 

very VH 
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An example of a rule is (R1 ): 

(Rl): if P(Rsk) is H and l(Rsk) is VH then Rsk is VH 

The interpretion of rule (Rl) is: 

If risk Rsk has a high probability of occurance and the impact of this risk is very high , 
then its value is also very high. (See the underlined values from Table 4) 

5. Application of the Fuzzy Model to Evaluate Risks in 
an E-testing Research Project 

The fuzzy model for risks evaluation is validated by applying it to a real research 
project. The project has the main goal of studying e-testing methods in e-learning 
environments and of discovering and improving those which has applicability in project 
management. Among research objectives, there are: education in a globalized society, e- 
learning in present times, tools for implementing e-learning systems, defining the place of e- 
testing in e-learning platforms, development of an e-testing model suitable for knowledge 
evaluation in project management field. Being a research project, the quality of the final 
results depends on the quality of the research idea and on budget constraints (see the 
highlighted elements from Figure 4). The risk of having a poor quality idea has attached a 
number of points (from 0 to 10), reflecting the level of innovation and applicability of the 
idea. The risk of having a low quality idea as the starting point of the project depends on the 
risk of accepting that idea. Over-budget risk is a well-known risk in projects, but in research 
projects it can add value to scientific results, thus enhancing the satisfaction level of the 
stakeholders. 

5.1. Formalization of Risk Evaluation Model for E-testing Research Project 

In order to compute the probability of "Lack of quality results" risk (notated "P(Rsk)"), 
low embedded quality of the idea is reflected in "LowQAIdea" variable and over-budget sum 
in "OverBudget" variable. Both variables are defined in Table 5. Inference rules for showing 
the effect of "LowQAIdea" and "OverBudget" changes on "Rsk" value are presented in Table 

6 . 


Table 5. Fuzzy Variables for C omp uting Risk in E-te sting Research Project 


Fuzzy Variable 
Name 

Variable 

Type 

Universe of Discourse 

Linguistic Grades 

OverBudget 

input 

[0,5] thousands of Euros 

VL, L, M, H, VH 

LowQAIdea 

input 

[0,10] points 

VL, L, M, H, VH 

P(Rsk) 

output 

[0,100]% 

VL, L, M, H, VH 


Table 6. Inference Rules for Computing "P(Rsk)" in E-testing Research Pro 


ect 


OverBudget / 
LowQAIdea 

VERY LOW 

LOW 

MEDIUM 

HIGH 

VERY HIGH 

VERY LOW 

somewhat 

M 

somewhat 

M 

L 

VL 

very VL 

LOW 

H 

H 

L 

VL 

VL 

MEDIUM 

VH 

H 

M 

L 

L 

HIGH 

VH 

VH 

H 

M 

M 

VERY HIGH 

VH 

VH 

VH 

M 

M 
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Membership functions of input variables are illustrated in Fig. 6 and 7 and the one 
for output variable in Fig. 8. 



Figure 6. Membership Functions for "Over-budget" variable 
in Evaluating Research Project Risk 



Figure 7. Membership Functions for "Low embedded quality of the idea" variable in 

Evaluating Research Project Risk 

( 25,0 5 ) 



( 10 , 0 ) # ( 50 , 0 ) 



Figure 8. Membership Functions for Fuzzy Output Variable in Evaluating 

Research Project Risk 
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5.2. Steps in Applying the Risk Evaluation Model in E-testing Research Project 

The fuzzy model transforms the input values to output value, in four steps. For 
exemplification purposes, two numerical values are considered as input values in risk 
evaluation model of an e-testing research project. 

Step 1 : Insert values for input variables 

OverBudget = 2.7 (the research needs an extra sum of 2,7 thousands Euros to be 

finished) 

LowQAIdea = 7 (the risk of having a low quality idea is 7/10, according to experts) 

Step 2: Classify crisp input values in suitable fuzzy set: they gain semantic meaning. 

According to Fig. 6, the over-budget sum can be high and very high. According to 
Fig. 7, the risk of a poor idea can be medium or high. 

Step 3: Establishes membership level of each input value to a fuzzy set; an input value can 
belong to one or two fuzzy sets, but in a different proportion, named "trust level"; this trust 
level is the Oy value of the intersection point between input singleton and fuzzy set. [9] Trust 
level is calculated using the following formula: 

trustLevel x (A) = b.y + m{x - b.x) (FI ) 

where a and b are two successive points from a fuzzy set, m is the slope of the line 
determined by these two points, A represents a fuzzy set and x represents an input value; 

For LowQAIdea: 

trustLevel 7 (M ) = 0.5 

trustLevel 7 (H ) = 0.33 

For OverBudget: 

trustLevel 2 7 ( H ) = 0.6 

trustLevel 2 7 ( VH ) = 0.13 
Following sentences are considered: 

(51) "An idea of medium quality is the starting point in e-testing research project." 

(52) "An idea of high quality is the starting point in e-testing research project." 

(53) "The over-budget used in e-testing research project is high." 

(54) "The over-budget used in e-testing research project is very high." 

According to above calculated trust levels, (SI) has a 50% value of truth, (S2) has 
33% value of truth, (S3) a 60% value of truth and (S4) a 1 3% value of truth. 

Step4: Apply inferences rules, in 4 stages. The resulted fuzzy sets are "cut" by a horizontal 
line. This line is determined by the minimum of the trust levels. Intersections between fuzzy 
sets and lines are obtained: this intersection has, usually, a trapezoidal shape. In the end, 
the reunion of "cut" fuzzy sets is made (see Fig. 8) 

Step5: Restrictors (very, somewhat ) are applied, if necessary. 

Step6: Two defuzzification methods are applied to the final fuzzy set. Two similar values for 
over-all risk in e-testing project should be obtained. 

In Centre of Gravity (COG) Defuzzification, the final fuzzy set is decomposed in 
simple shapes: triangles, rectangles and trapezes, as shown in Fig. 9. For calculating the 
probability of risk occurrence, the following formula is used: 
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H center fi g (i) area fi g (i) 

P(Rsk) COG = (F2) 

L area fi g (i) 

i 

where fig is the shapes vector, center^ {i) is the center of gravity of a figure and area fig{i) is 
the area of a figure. 



(1 0 n 0) Fig-1 Fig.2 Fig.3 Fig.4 Fig.5 


Figure 9. COG Defuzzification in Risk Evaluation Model 


According to COG method, the probably of risk "Lack of quality results" for the 
considered input values is: 

13.33*1.25 + 20*5 + 28.05*2.05 + 38.33*5.53 + 47.8*0.54 
P{Rsk) COG = 


P{Rsk) COG = 


1.25 + 5 + 2.05 + 5.53 + 0.54 
16.67 + 100 + 57.7 + 212 + 25.81 _ 

14.37 ^ 


P(Rsk) COG = 28.68% 

In Middle of Maximum (MOM) Defuzzification, the following formula is used: 


P(Rsk) MOM 


^ locMax * locMaxf 

i 

y, locMax: 


(F3) 


where locMax ■ is the Oy value of a local maximum (see Fig. 10), locMax* is the Ox value 
of a local maximum point and locMax is the vector of local maximum points. 
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According to MOM method, the probably of risk "Lack of quality results" for the 
considered input values is: 

0 5*15 + 0 5*25 

P(Rsk) MOM = - Q5 + o5 =* P(Rsk) MOM = 20% 

The two defuzzification methods revealed similar results (28,68% and 20%) : the 
applied model is validated. The membership function of output variable is analyzed (see Fig. 
8) and the conclusion is that the "Lack of quality results" risk has a low probability of 
occurance for an e-testing research project. 

5.3. Software for Risk Evaluation based on Fuzzy Model 

The fuzzy model for analyzing the over-all risk in the e-testing research project was 
used to develop a fuzzy system. A Windows based application was created. The interface is 
intuitive (see Fig. 11): the end user has to insert the value of "over-budget" risk and of "low 
embedded quality of the idea" risk and will obtain the probability of over-all risk occurrence 
in the e-testing research project. 


> 


Figure 1 1. Software Product for Risk Evaluation Model in 

The algorithm used to create the software reflects entirely the steps described in 
current paper. Three specific classes are used to implement the algorithm (see Fig. 12): 

• FuzzySystem: it contains the inference rules; 

• FuzzySet: it contains defuzzification methods, union, intersection, restrictor 
functions; 

• FuzzyPoint: elements of fuzzy sets; 

FuzzySystem class has to be changed, in order to create other fuzzy systems. 
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Figure 12 . Class Diagram for Fuzzy Risk Evaluation System 


6. Conclusions 


The proposed model offers an easy-to-use tool for risk evaluation in research 
projects. The model lies on fuzzy inference. The knowledge base used by fuzzy rules is built 
on causal and cognitive maps of risks. Although research projects are known for their high 
level of risk, very few dedicated risk systems were developed especially for them. Therefore, 
the fuzzy model for risk evaluation in research projects is an innovative instrument which can 
be used to forecast project failure: stakeholders can save money, time, effort, without giving 
up the quality of predictions. The model was used to develop a software system for 
evaluating risk in an e-testing research project, so its applicability was validated. The system 
can be further developed to evaluate all risks from the map, not only the one from the 
highest level, as it does now. 
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